Picture coding method and picture decoding method

ABSTRACT

In a picture coding method for generating a coded signal corresponding to each picture by coding a plurality of coded signals, a switching picture which is capable of switching a plurality of coded signals and subsequent pictures of the switching picture can refer to only a group of pictures of the same time in the coded signals. More specifically, the case where picture numbers of an adjacent picture of an S picture and the S picture are not continuous is not considered as an error.

TECHNICAL FIELD

The present invention relates to a picture coding method for efficientlycompressing moving picture signals by using correlation betweenpictures, a picture decoding method for decoding the signals correctly,and a recording medium on which a program for executing these methods byusing software is recorded.

BACKGROUND ART

The multimedia era has arrived recently in which sound, pictures andother pixel values are integrated into one media, and conventionalinformation media as communication tools like newspapers, magazines, TV,radio and telephone are regarded as the targets of multimedia.Generally, multimedia is a form of the simultaneous representation ofnot only characters but also graphics, sound, and especially pictures.In order to handle the above-mentioned conventional information media asmultimedia, it is a requisite to represent the information digitally.

However, it is unrealistic to directly process a huge amount ofinformation digitally by using the above-mentioned conventionalinformation media because, when calculating the data amount of eachinformation medium mentioned above as a digital data amount, the dataamount per character is 1-2 bytes while that of sound per second is notless than 64 kbits (telephone speech quality) and that of movingpictures per second is not less than 100 Mbits (present TV receivingquality). For example, a TV telephone has already become commerciallypractical thanks to Integrated Services Digital Network (ISDN) with atransmission speed of 64 kbps-1.5 Mbps, but it is impossible to transmitpictures of a TV camera as they are using ISDN.

That is why an information compression technique is necessary. Forexample, a moving picture compression technique standard of H.261 orH.263, which is internationally standardized by the InternationalTelecommunication Union-Telecommunication Standardization Sector(ITU-T), is used for TV telephones. Also, it is possible to storepicture information with sound information in ordinary music compactdiscs (CDs) by using the information compression technique of the MPEG-1standard.

Here, Moving Picture Experts Group (MPEG) is an international standardto digitally compress moving picture signals, and MPEG-1 is the standardto compress moving picture signals to 1.5 Mbps, that is, to compress TVsignal information to about one-hundredth of the original size of theinformation. Also, the quality which satisfies the MPEG-1 standard is amedium level that can be realized at a transmission rate of about 1.5Mbps. MPEG-2 is thus standardized in order to satisfy the need forhigher picture quality, and MPEG-2 compresses moving picture signals to2-15 Mbps.

Presently, the work group (ISO/IECJTC1/SC29/WG11), which standardizedMPEG-1 and MPEG-2, has standardized MPEG-4 with a higher compressionrate. MPEG-4 introduced not only efficient coding at a low bit rate, butalso a powerful error-resist technique that lessens subjective picturedeterioration in case a transmission error occurs. Also, as a picturecoding system for the next generations, ISO/IEC and ITU-T are jointlyworking for the standardization of Joint Video Team (JVT). The standardcalled Joint Model 2 (JM2) is presently the latest version.

The picture for intra predictive coding without any reference picture iscalled Intra Coded Picture (I picture). In addition, the picture forinter predictive coding with a reference picture is called PredictiveCoded Picture (P picture). Also, the picture for inter predictive codingin which two reference pictures are referred to simultaneously is calledBi-predictive Coded Picture (B picture).

The term “picture,” as used herein, is a term representing one picture.In a progressive picture, a picture means a frame, but in an interlacepicture, a picture means a frame or a field. An “interlace picture,” asused herein, means a frame that is composed of two fields with a slighttime lag. In the coding and decoding processes of interlace pictures, itis possible to process a frame as it is, as two fields, or by each blockin a frame in a frame-by-frame structure or in a field-by-fieldstructure.

In JVT, it is possible to choose an arbitrary picture as a forwardreference picture from a plurality of pictures, unlike conventionalmoving picture coding. Also, a system to switch coded bit streams atspecific pictures, that is, Switching Coded Pictures (S pictures), hasbeen introduced. (There are SI pictures and SP pictures in S pictures,and these are the pictures for intra predictive coding or interpredictive coding, respectively.)

The S picture system is to guarantee that streams after S Pictures canbe decoded correctly in the case of switching from stream to stream justbefore S pictures. Also, it is possible to switch streams at a serversuch as a moving picture distribution server according to thecommunication capacity of receiving terminals or the preference ofreceivers.

In the conventional picture coding method or conventional picturedecoding method, S pictures are introduced so that (1) it can choose anarbitrary picture as a forward reference picture from a plurality ofpictures, and (2) it can also switch pictures at specific pictures.Regrettably, in spite of the introduction of these two techniques,problems that occur when these two techniques are combined have not yetbeen adequately resolved. In reality, it is difficult to use both thetechniques together because of the problems shown below.

FIG. 1 is an illustration showing the relationship between pictures andpicture numbers (PN) when coding an input picture signal (VIN). The samepicture signal is coded at different picture rates (the number ofpictures per second) to make Streams 1, 2 and 3. Picture numbers (PN)are numbers to identify coded pictures. In JM2, pictures to be referredto as reference pictures in the following coding are assigned numberswhich are incremented by 1. To simplify the explanation, the example ofFIG. 1 shows only the case where all of the pictures in each stream arereferred to as reference pictures in the following coding, and thepicture numbers are always incremented by 1. Pictures that are notreferred to in the following coding are unrelated to the increase ordecrease in the picture numbers, and are not stored in a memory.Therefore, an explanation as to the pictures that are not referred to inthe following coding is omitted because the pictures are unrelated tothe following explanation of operations.

As shown in FIG. 1, at the time of t3, the diagonally shaded picturesare coded as S pictures. FIG. 2 is a diagram showing picture numbers(PN) of the pictures to be stored in the reference picture memory whencoding or decoding S pictures.

FIG. 2 shows pictures which are stored in the reference picture memory(Mem) and their respective positions. In the reference picture memory(Mem), pictures in the left position are newer in time than pictures inthe right position. At the time of predictive coding, the same picturesmust be referred to in coding and decoding. When it is possible tochoose each reference picture from a plurality of reference pictureslike in the JM2, it is necessary to specify which pictures are referredto.

As described below, there are two methods to show reference pictures,and JM2 uses these two methods properly according to the intendedpurposes.

{circle around (1)} Clearly expressing how many pictures there are priorto a newer picture; and

{circle around (2)} Clearly expressing a reference picture by a picturenumber (PN).

In order to correctly code S pictures and the following pictures and todecode these pictures correctly at the time of decoding when switchingstreams at S pictures, the contents of the reference picture memory(Mem) must be the same in every case of switching streams at S pictures.

However, as shown in the illustration of FIG. 2, which shows the picturenumbers (PN) of pictures to be stored in the reference picture memory(Mem), the contents of the reference picture memory (Mem) are not thesame at the start of coding or decoding an S picture in each stream.Unless such a conventional method is improved, it is impossible to usethe coding method to choose reference pictures from the referencepicture memory (Mem) in combination with the S picture system forswitching streams.

The present invention aims at solving all of the above-mentionedproblems. In particular, the present invention aims to make the Spicture system usable in combination with the other coding method tochoose reference pictures in a reference picture memory (Mem), therebyproviding picture coding and decoding methods that improve compressionrates in the above-mentioned combined coding method using S pictures.

SUMMARY OF THE INVENTION

The present invention provides the following inventions in order toresolve the above-mentioned problems with the conventional art.

The first invention is a picture coding method for specifying areference picture that is stored in a memory by using a picture numberfor reference and generating a coded stream of moving pictures. Thepicture coding method comprises a picture number coding step of codingpicture numbers corresponding to the current pictures, a coding step ofcoding the current pictures, a step of making all of the pictures storedin the memory except for the current pictures released for referenceafter the picture coding step, a picture number initializing step ofinitializing the picture numbers of the current pictures in the memory,and an all picture release information coding step of coding all picturerelease information so as to instruct a picture decoding apparatus torelease all of the pictures that have already been stored in the memoryexcept for the current pictures.

The second invention is a picture decoding method for specifying areference picture that is stored in a memory by using a picture numberfor reference and decoding the coded stream of moving pictures. Thepicture decoding method comprises an all picture release informationdecoding step of checking and decoding an all picture releaseinformation, which means releasing all of the pictures in a coded streamstored in a memory except for the current pictures to be decoded, adecoding step of decoding the current pictures in the coded stream, astep of releasing all of the pictures that are stored in the memoryexcept for the current pictures according to the decoded all picturerelease information after the decoding step, and a picture numberinitializing step of assigning an initialized picture number to thecurrent picture in the memory.

The third invention is a picture coding method for coding a plurality ofpicture signals and generating coded signals corresponding to therespective pictures, wherein a switching picture which is capable ofswitching a plurality of coded signals and the following pictures of theswitching picture can refer to only a group of pictures of the same timein the coded signals.

The fourth invention is a picture decoding method for decoding a codedsignal, wherein information on pictures which is released for referencebefore a switchable switching picture is decoded, decoded pictures in areference picture memory are released based on the decoding result, andafter the switching picture, a coded signal is decoded by referring to areference picture that is not released.

The fifth invention is a picture coding method for coding a plurality ofpicture signals and generating coded signals corresponding to respectivepictures, wherein the picture coding method has a step of changing apicture number of a switching picture which is capable of switching aplurality of coded signals into the same value at a respective one ofcoded signals.

The sixth invention is a picture decoding method for decoding a codedsignal comprising a step of changing a picture number of a referencepicture into the same value in a switchable coded signal at the time ofswitching coded signals at a switchable switching picture.

As mentioned above, with the picture coding method and the picturedecoding method in the present invention, it is possible to use thefeature of S pictures and a coding method for choosing a referencepicture in a reference picture memory in combination, which makes itpossible to provide a picture coding method and a picture decodingmethod to improve compression rates even when using S pictures in thecoding method, and thus these methods are highly practical.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of the relationship between pictures andpicture numbers (PN) when coding an input picture signal (Vin).

FIG. 2 is a diagram showing picture numbers (PN) of pictures to bestored in a reference picture memory (Mem) when coding and decoding Spictures.

FIG. 3 is an illustration of picture numbers (PN) of pictures to bestored in the reference picture memory (Mem).

FIGS. 4A, 4B and 4C are flowcharts of how to code and decode informationon controlling pictures to be stored in the reference picture memory(Mem) in the picture coding method and the picture decoding method ofthe present invention.

FIGS. 5A and 5B are illustrations of picture numbers (PN) of pictures tobe stored in the reference picture memory (Mem).

FIGS. 6A, 6B and 6C are flowcharts of how to code and decode informationon controlling pictures to be stored in the reference picture memory(Mem) in the picture coding method and the picture decoding method ofthe present invention.

FIG. 7 is an illustration of the relationship between pictures andpicture numbers (PN) when coding an input picture signal (Vin) of thepresent invention.

FIGS. 8A and 8B are flowcharts of how to decode information oncontrolling pictures to be stored in the reference picture memory (Mem)in the picture decoding method of the present invention.

FIG. 9 is a block diagram showing the structure of picture codingapparatus of the present invention.

FIGS. 10A, 10B, 10C and 10D are diagrams showing data structure examplesof coded signals Str of the present invention.

FIG. 11 is a block diagram showing the structure of a picture decodingapparatus of the present invention.

FIG. 12 is an illustration of the relationship between pictures andpicture numbers (PN) for coding an input picture signal (Vin) of thepresent invention.

FIG. 13 is a flowchart showing a coding method by giving a picturenumber to each picture of each stream in the present invention.

FIG. 14 is a flowchart showing a decoding method in a seventh embodimentof the present invention.

FIG. 15 is a diagram showing a memory structure in the seventhembodiment.

FIGS. 16A and 16B are flowcharts showing a coding method in the seventhembodiment.

FIGS. 17A and 17B are flowcharts showing another coding method in theseventh embodiment.

FIG. 18 is a flowchart showing another coding method in the seventhembodiment.

FIG. 19 is a block diagram showing a structure of a coding apparatus inan eighth embodiment of the present invention.

FIG. 20 is a block diagram showing a structure of another codingapparatus in the eighth embodiment.

FIGS. 21A and 21B are flowcharts showing a decoding method in a tenthembodiment of the present invention.

FIG. 22 is a block diagram showing a structure of a decoding apparatusin an eleventh embodiment of the present invention.

FIGS. 23A and 23B are flowcharts showing the processing for making codedsignals and decoding the coded signals.

FIG. 24 is a block diagram showing a structure of a picture codingapparatus realizing a coding method in a twelfth embodiment of thepresent invention.

FIG. 25 is a block diagram showing a structure of a picture decodingapparatus realizing the decoding method in the twelfth embodiment.

FIGS. 26A, 26B and 26C are illustrations relating to a recording mediumstoring a program for realizing the picture coding methods and picturedecoding methods in the first to the twelfth embodiments using acomputer system.

FIG. 27 is a block diagram showing the whole structure of a contentsupplying system realizing a content distribution service concerning thepresent invention.

FIG. 28 is a diagram showing an example of cellular phones as relatingto the present invention.

FIG. 29 is a block diagram showing the structure of the same cellularphones of FIG. 28.

FIG. 30 is a diagram showing the structure of a digital broadcastingsystem concerning the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present embodiments of the present invention will be explained belowwith reference to FIGS. 3-30.

First Embodiment

FIG. 3 is an illustration showing picture numbers (PN) of pictures to bestored in the reference picture memory (Mem). The difference betweenFIG. 3 and FIG. 2 showing picture numbers (PN) of pictures to be storedin the reference picture memory (Mem) will be explained below.

It has already been explained that, after switching coded signals whencoding and decoding S pictures, the content of the reference picturememory (Mem) is not the same. Therefore, in the coding and the decodingmethods of the present invention in FIG. 1 showing the relationship ofpictures and their picture numbers (PN) when coding an input picturesignal (Vin), only pictures at the times of t0, t1 and t2 that areexactly the same respectively in all of the streams are stored in thereference picture memory (Mem), while the other pictures are deleted inthe reference picture memory (Mem) before coding and decoding Spictures. FIG. 3 shows the result of this processing as an illustrationshowing the picture numbers (PN) of the pictures to be stored in thereference picture memory (Mem).

As shown in FIG. 3, explaining picture numbers of pictures to be storedin the reference picture memory (Mem), when the method of “Clearlyexpressing how many pictures there are prior to a newer picture” isemployed to specify reference pictures in coding and decoding, it ispossible to code and decode the pictures correctly because exactly thesame pictures of the same time are referred to in any case of stream 1,2 or 3.

FIG. 4 is a flow chart showing coding and decoding methods ofinformation concerning picture control of pictures to be stored in thereference picture memory (Mem) in the picture coding decoding methods ofthe present invention.

FIG. 4A, as a flowchart of the coding method, shows how to realize themethod of the operation explained in FIG. 3, and the coding and decodingmethods of the necessary information for realizing the operation.

In Step 0, pictures at the same points of time in a plurality of codinginformation (streams) are chosen. In Step 1, it is possible to codedelete information showing the deletion of the other pictures which arenot chosen in Step 0. In Step 2, pictures which are not chosen in Step 0are deleted from the reference picture memory (Mem). Up to this point,as shown in FIG. 3, it is possible to realize the storage condition inthe reference picture memory (Mem) so as to realize decodable streamseven after switching coded signals.

Also, it is possible to change the order of Step 1 and 2, and if theorder is changed, the flowchart of the picture coding method shown inFIG. 4B is used.

Decoding the delete information which is coded according to the codingmethod shown in the flowchart of FIG. 4A by using the decoding methodshown in the flowchart of FIG. 4C makes it possible to realize thestorage condition of the reference picture memory (Mem) to realizedecodable streams by using the picture decoding method even afterswitching coded signals as shown in FIG. 3.

By decoding the delete information in Step 5, it is possible to showpictures that are not the same-time pictures in a plurality of codinginformation (streams). These pictures are the rest of the pictures thatare chosen in Step 0 of FIG. 4A as pictures of the same time. Next, inStep 6, pictures that are chosen in Step 5 are deleted in the referencepicture memory (Mem). Specifically, when pictures that are stored in thereference picture memory (Mem) are deleted (or erased), pictures to bedeleted are assigned IDs (identification information) such as “release”which prohibits such pictures from being used as reference pictures. Forthat purpose, a picture decoding unit (PicDec) and a picture encodingunit (PicEnc) always check whether or not IDs of “release” are assignedeach time these units refer to pictures that are stored in the referencepicture memory (Mem). Only pictures that are not assigned IDs of“release” are referred to by these units, while pictures that areassigned IDs of “release” are not referred to by these units. Likewise,pictures in the reference picture memory (Mem) are deleted (or erased)in the following embodiments. Of course, as this deletion method is anexample, it goes without saying that it is possible to delete theabove-mentioned picture data in the reference picture memory (Mem) byactually deleting or erasing these data. Up to this point, it ispossible to realize the storage condition in the reference picturememory (Mem) so as to realize decodable streams even after switching thecoded signals as shown in FIG. 3.

Second Embodiment

FIG. 5A is an illustration showing picture numbers (PN) of pictures tobe stored in the reference picture memory (Mem). The difference betweenFIG. 5A and FIG. 3 showing picture numbers (PN) of pictures to be storedin the reference picture memory (Mem) is whether or not picture numbers(PN) in the reference picture memory (Mem) are the same.

Since not only the times of pictures that are stored in the referencepicture memory (Mem), but also the picture numbers (PN) in each streamin the reference picture memory (Mem) are the same in the time frame, itis possible to use the method of “Clearly expressing reference picturesby picture numbers (PN)” when specifying reference pictures in codingand decoding. Thus, it becomes possible to code and decode the picturescorrectly because exactly the same pictures of the same time arereferred to in any case of stream 1, 2 or 3.

It is possible to realize this by replacing the picture numbers ofpictures that are stored in the reference picture memory (Mem) with anew same picture number, and by coding and decoding the information forreplacing the old picture numbers with the new one before coding anddecoding S pictures.

In addition, there is a need to match picture numbers (PN) of S picturesin any of the streams because the same picture numbers (PN) must be usedwhen storing S pictures the next time.

FIG. 6 is a flowchart showing coding and decoding methods of informationfor controlling pictures to be stored in the reference picture memory(Mem) according to the picture coding and decoding methods of thepresent invention, and FIG. 6 shows the realization methods of theoperation explained in FIG. 5A and the coding and decoding methods ofinformation that is necessary for the realization.

In Step 10, the maximum value of picture numbers (PN) (“8” in theexample of FIG. 5A.) of pictures in the coded signals to be switched inthe reference memory (Mem) are detected. In Step 12, information forreassigning picture numbers (PN) of each picture that is stored in thereference picture memory (Mem) is coded with reference to the maximumvalue of picture numbers (PN). Also, as the need arises, picture numbers(PN) to be assigned to the next S pictures are coded. As stream 3 inFIG. 5A is the same as stream 3 in FIG. 3, there is no need to reassignpicture numbers of pictures in stream 3. Therefore, picture numbers arereassigned to necessary pictures only, and thus, only information onnecessary reassignment needs to be coded in Step 11. Lastly, picturenumbers which are shown by the coded information in Step 11 isreassigned in Step 12. Up to this point, as shown in FIG. 5, the storagecondition in the reference memory (Mem) for realizing decodable streamseven after switching the coded signals is realized.

Also, as the picture number (PN) of S pictures are 12, in order to makepicture numbers (PN) continuous after coding and decoding these Spictures, it is possible to use picture number 11, that is, the picturenumber (PN) immediately before an S picture (immediately before an Spicture of stream 1 in FIG. 1) as shown in FIG. 5B. In this case,picture numbers (PN) always increase in the process of coding anddecoding as the picture number (PN) of the S picture is 12, which ismore effective because the error check function to regard a decrease inpicture numbers (PN) as an error is also realized.

FIG. 7 is an illustration showing the relationships between pictures andpicture numbers (PN) when coding an input picture signal (VIN) of thepresent invention. FIG. 7 is an example of the reassignment of picturenumbers (PN) by using the method explained in FIG. 5B, where all thepicture numbers of S pictures are 12. Therefore, it is clear that allthe pictures after S pictures are correctly decodable even afterswitching streams at S pictures because pictures in the reference memory(Mem) are identical irrespective of streams when coding and decoding Spictures. Also, it is possible to change the operational order of Step11 and 12, and in this case, a flowchart of picture coding method shownin FIG. 6B is used.

Decoding the coded delete information in FIG. 6A shown in the flowchartof the coding method by using the decoding method shown in the flowchartin FIG. 6C of the decoding method makes it possible to realize thestorage condition of the reference picture memory (Mem) so as to realizedecodable streams using the picture decoding method even after codedsignals are switched as shown in FIG. 5A.

Decoding the information on the reassignment of picture numbers (PN) inStep 15 makes it possible to specify the pictures which are necessaryfor the reassignment of picture numbers (PN) and the method. Next, inStep 16, picture numbers (PN) of pictures in the reference picturememory (Mem) are reassigned based on the pictures which are decoded inStep 15, which thus requires the reassignment of picture numbers (PN)and the reassignment method. Up to this point, it is possible to realizethe storage condition in the reference picture memory (Mem) so as torealize decodable streams even after switching the coded signals asshown in FIG. 5.

While this second embodiment explains the effectiveness in combinationwith the first embodiment, as the second embodiment can solely realizethe merit of coding and decoding correctly in the case of “Clearlyexpressing reference pictures by picture numbers (PN)”, it is possibleto use only the second embodiment instead of the second embodiment incombination with the first embodiment if the effect of the secondembodiment is sufficiently effective.

Third Embodiment

FIG. 8 is another embodiment for realizing an illustration of picturenumbers (PN) of pictures to be stored in the reference picture memory(Mem) in FIG. 5.

Picture types are identified by picture type information (PicType).Therefore, when a picture is identified as an S picture that can changestreams by the picture type information (PicType), making a rule forreassigning picture numbers (PN) of pictures in the reference picturememory (Mem) to coincide with picture numbers (PN) of S pictures makesit possible to omit coding and decoding the information on reassignmentmethod for each picture number (PN) of pictures in the reference picturememory (Mem).

The operation shown in FIG. 8A will be explained below. In Step 20,picture numbers (PN) of pictures are obtained by decoding coded signals.The picture type information (PicType) of the pictures is obtained inStep 21. When the picture type information (PicType) is proved to be Spictures, picture numbers (PN) of pictures in the reference picturememory (Mem), when they coincide with picture numbers (PN) of Spictures, are reassigned based on a specified method in Step 22. Up tothis point, as shown in FIG. 5, it is possible to realize the storagecondition in the reference picture memory (Mem) so as to realizedecodable streams even after switching coded signals.

Also, it is possible to change the order of Step 21 and 22, where if theorder of Step 21 and 22 is changed, the flowchart of the picture codingmethod shown in FIG. 8B is used.

Also, it is possible to code and decode only part of reassignmentinformation of picture numbers (PN) (which cannot be represented by therule for reassigning picture numbers (PN) in the reference picturememory (Mem) when they coincide with picture numbers (PN) of S pictures)in Step 11 and 15 in FIG. 6 by combining the illustration of FIGS. 6 and8 concerning picture numbers (PN) of pictures to be stored in thereference picture memory (Mem) in FIG. 5.

Fourth Embodiment

FIG. 9 is a block diagram showing the structure of the picture codingapparatus of the present invention. FIG. 9 is a block diagram concerningthe picture coding apparatus of the present invention, and is an examplefor realizing the picture coding method in the first embodiment and thesecond embodiment.

A picture number generating unit (PNGen) generates picture numbers (PN).Picture numbers (PN) are IDs to identify the pictures that are stored inthe reference picture memory (Mem), and each picture that is stored inthe reference picture memory (Mem) is assigned an exclusive picturenumber (PN). Usually, picture numbers (PN) are incremented by 1 eachtime a picture is stored in the reference picture memory (Mem). When apicture number (PN) that is received by a picture decoding apparatus isincremented by 2 or more, the picture decoding apparatus can find that apicture to be stored is missing because of a transmission line error,and thus can correct the error and make the error less conspicuous.

A maximum picture number checking unit (MaxPN) compares other codedsignal picture numbers (OtherPN) and picture numbers (PN) which aregenerated in the picture number generating unit (PNGen), detects themaximum value of picture numbers (PN), notifies a variable length codingunit (VLC) and the picture number generating unit (PNGen) of the maximumvalue of picture numbers (PN), and initializes picture numbers (PN) tobe generated in the picture number generating unit (PNGen) by using themaximum value of picture numbers (PN). Other coded signal picturenumbers (OtherPN) are picture numbers of pictures in a different streamin parallel with the pictures to be coded. Consequently, after that, thepicture number generating unit (PNGen) starts to output picture numbers(PN) which are larger than the maximum value of the picture numbers(PN).

A coded picture time comparing unit (TimeCmp) compares the frame time ofeach picture in the input picture signal (Vin) that is coded so far andthe frame time of each picture that is coded as other coded signals(streams), and notifies a picture deleting unit (PicDel) of pictureinformation concerning the frame time that is coded in all the streams.

When picture type information (PicType) shows that a next picture is anS picture, the picture deleting unit (PicDel) orders a reference picturememory (Mem) to delete the pictures outside the time frame in all thestreams that are stored in the reference picture memory (Mem) based onthe information which is notified by the coded picture time comparingunit (TimeCmp), and the picture deleting unit (PicDel) notifies thevariable length coding unit (VLC) of the same information at the sametime.

A picture encoding unit (PicEnc) refers to the pictures in the referencepicture memory (Mem), codes the input picture signal (Vin) includingfrequency conversion and quantization as a picture type that is shown inthe picture type information (PicType), and sends the result to picturedecoding unit (PicDec) and variable length coding unit (VLC). A picturedecoding unit (PicDec) performs an inverse quantization and frequencyconversion of the coding result in a picture encoding unit (PicEnc) aspicture types that are shown in the picture type information (PicType),and stores the picture types as picture numbers (PN) in the referencepicture memory (Mem) so as to refer to the picture types in thefollowing picture coding process.

The variable length coding unit (VLC) makes the coded result in thepicture encoding unit (PicEnc) into variable length codes so as to makea bit stream, codes the information which is needed for decoding, thatis, the information for deleting pictures that are stored in thereference picture memory (Mem) which is notified by the picture deletingunit (PicDel), the maximum value of the picture numbers (PN) and thepicture numbers (PN) mentioned above, so as to output the information ascoded signals (Str). The variable length coding unit (VLC) also codesinformation which is notified by the picture deleting unit (PicDel) andinformation for reassigning picture numbers (PN) of pictures that arestored in the reference picture memory (Mem) based on the method shownin the second embodiment.

FIG. 10 shows a structural example of coded signals (Str) in the presentinvention. Each data in FIG. 10A will be explained below.

First, picture numbers (PN) are coded. Next, the maximum picture number(PN) to be reassigned, information for deleting pictures that are storedin the reference picture memory (Mem) and information for reassigningpicture numbers that are stored in the reference picture memory (Mem)are coded. Then, picture type information (PicType) and picture codeddata which is outputted by the picture encoding unit (PicEnc) arelocated.

As FIG. 10A is simply an example of data location, it is possible tochange the order of data as shown in FIG. 10B so as to perform picturecoding.

It is also possible to prepare the picture coding apparatus consistingof the above-mentioned units so as to realize the picture coding methodshown in the first and the second embodiments.

Fifth Embodiment

FIG. 11 is a block diagram showing the structure of the picture decodingapparatus of the present invention. FIG. 11 is a block diagramconcerning the picture decoding apparatus of the present invention, andis an example of a picture decoding apparatus that realizes the first,the second and the third embodiments. The function of the picturedecoding apparatus of the present invention will be explained below.

A variable length decoding unit (VLD) decodes coded signals (Str), andoutputs various information (such as an order for deleting pictures thatare stored in the reference picture memory (Mem), picture typeinformation (PicType), picture numbers (PN), information for reassigningpicture numbers (PN) and picture data.)

The order for deleting pictures that are stored in the reference picturememory (Mem) is obtained in a variable length decoding unit (VLD) and isfirst sent to a picture deleting unit (PicDel). The picture deletingunit (PicDel) deletes the specified pictures that are stored in thereference picture memory (Mem).

Picture type information (PicType) which is obtained in the variablelength decoding unit (VLD) is sent to a picture decoding unit (PicDec)so as to show the decoding method.

Picture numbers (PN) which are obtained in a variable length decodingunit (VLD) are sent to the reference picture memory (Mem) as picturenumbers (PN) when storing the pictures that are decoded in a picturedecoding unit (PicDec).

Information for reassigning picture numbers (PN) of pictures that arestored in the reference picture memory (Mem) which is obtained in thevariable length decoding unit (VLD) is sent to a picture number changingunit (PNchg). The picture number changing unit (PNchg) follows thedirections and reassigns picture numbers (PN) of pictures that arestored in the reference picture memory (Mem). To be more specific, thepicture number changing unit (PNchg) reads out picture numbers (PN) ofpictures that are stored in the reference picture memory (Mem),reassigns the values of the picture numbers (PN) that are read out, andthen writes the new picture numbers (PN) in the reference picture memory(Mem).

In the picture decoding unit (PicDec), picture data obtained in thevariable length decoding unit (VLD) are decoded in a suitable decodingmethod for the specified picture type shown as picture type information(PicType). In other words, I pictures are decoded without referring topictures in the reference picture memory (Mem), while P pictures and Bpictures are decoded by referring to pictures in the reference picturememory (Mem). The decoded pictures obtained in this manner are stored inthe reference picture memory (Mem) and are outputted as decoded picturesignals (Vout).

Up to this point, it is possible to prepare the picture decodingapparatus consisting of the above-mentioned units so as to realize thepicture decoding method shown in the first, the second, and the thirdembodiments.

Sixth Embodiment

In the picture coding apparatus shown in the first to the fifthembodiments, when switching streams at S pictures, the picture numbersof the pictures before the switchable pictures are switched so as tomake the picture numbers continuous with the picture numbers of theswitchable pictures. In this sixth embodiment, picture numbers areswitched at the switchable pictures.

As to coding of a plurality of streams having a different picture rateor a bit rate, or a different structure, here is an example of a streamswitching method that enables coding after switching coded pictures froma picture under coding in a stream to a picture in another stream. Forconvenience of explanation, the simpler phrase of “switching streams” isused below.

In addition, in this sixth embodiment, whether or not pictures to becoded should be stored in the reference memory is judged based on theincrement in picture number between pictures to be coded and picturesforwardly adjacent to the pictures to be coded (Simply, “the precedingpicture” is used below.) in the coding order. To be specific, when anincrement in picture number between a preceding picture and a picture tobe coded is 1, the picture to be coded is stored in the referencememory. On the other hand, when the picture number of the pictures to becoded is the same as the picture number of the preceding pictures, thecurrent picture is not stored in the reference memory.

The processing of switching picture numbers (PN) of switchable pictureswill be explained concretely with reference to FIG. 12.

FIG. 12 is a diagram showing an example of the relationship betweenpictures and picture numbers (PN) when an input picture number (Vin) iscoded. An identical picture signal is coded at different picture ratesto make streams of 1, 2 and 3. In FIG. 12, pictures are locatedaccording to the cording order in each stream.

In stream 1, a picture number (PN) is assigned to each picture so thatthe picture numbers are incremented by 1. Also, in stream 2, there arepictures which are assigned picture numbers (PN) that are incremented by1 and pictures which are assigned the same picture numbers (PN) as theprecedent pictures. Also, in stream 3, a picture number (PN) is assignedto each picture so that the picture numbers are incremented by 1 similarto stream 1.

Therefore, as the picture numbers are incremented by 1 in streams 1 and3, pictures to be coded are stored in the reference memory. In stream 2,pictures which are assigned picture numbers (PN) in a way that thepicture numbers are incremented by 1 are stored in the reference memory,and pictures where assigned the same picture numbers (PN) as theprecedent picture(s) are not stored in the reference memory.

Also, pictures which are assigned the picture number of “0” in streams1, 2 and 3 are pictures that are to be displayed at the time of to.Likewise, the groups of pictures listed below are pictures that are tobe displayed at the same time: Picture F14 in stream 1, picture F22 instream 2 and picture F31 in stream 3 are pictures that are to bedisplayed at the time of t1; picture F18 in stream 1, picture F24 instream 2 and picture F32 in stream 3 are pictures that are to bedisplayed at the time of t2; picture F112 in stream 1, picture F26 instream 2 and picture F33 in stream 3 are pictures that are to bedisplayed at the time of t3; and picture F117 in stream 1, picture F215in stream 2 and picture F34 in stream 3 are pictures that are to bedisplayed at the time of t4. Note that pictures F112, F26 and F33correspond to S pictures in the first and second embodiments.

In FIG. 12, streams are switched by way of picture BP1 and BP2 thatexist between the picture before switching and the picture afterswitching, and both of the pictures BP1 and the BP2 are switchingpictures which are to be coded in a way so that they have the same timeas their precedent pictures in the respective switching streams.

For example, in the case where picture F026 in stream 2 (a picture inthe stream before switching) is switched to picture F113 in stream 1 (apicture in the stream after switching), switching picture BP1 thatexists between F26 and F113 is used as a picture at the time of t3. Inthis case, the picture number of switching picture BP1, which is aswitching picture, is changed to “12” so as to make the numbercontinuous with the picture number 13 of picture F113 in the streamafter switching.

Similarly, likewise, in the case where picture F32 in stream 3 (apicture in the stream before switching) is switched to picture F213 instream 2 (a picture in the stream after switching), switching pictureBP2 that exists between F32 and F213 is used as a picture at the time oft3. In this case, the picture number of switching picture BP2, which isa switching picture, is changed so as to make the number continuous withthe picture number 13 of picture F213 in the stream after switching.

In this way, by assigning picture numbers (PN) of switching pictures tomake the numbers continuous with the picture numbers (PN) of pictures inthe stream after switching, the picture numbers (PN) of pictures in thestream after switching are changed so as to be identical in any case ofcoded pictures within each stream or switching streams.

Next, the processing of assigning picture numbers (PN) in the case ofswitching streams will be explained below.

FIG. 13 is a flowchart showing a coding method after assigning picturenumbers (PN) to respective pictures in the streams in FIG. 12.

In step 1401, it is judged whether or not each of the pictures to becoded is an S picture. When the current pictures are S pictures, thepicture numbers (PN) of the current pictures are changed to the initialvalues of M in step 1402. When the current pictures coded are not Spictures, the picture numbers (PN) of the current pictures are notchanged.

In step 1403, it is judged whether or not each of the pictures to becoded is a next picture of an S picture. When the current pictures arethe next pictures of S pictures, it is judged in step 1404 whether ornot each of the S pictures is stored in the memory. When the currentpictures are not the next pictures of S pictures, it is judged in step1405 whether or not each of the current pictures is stored in thememory.

When S pictures are judged to be stored in the memory in step 1404, thepicture number “M” is incremented by 1 to make M+1 in step 1406, and theincremented picture numbers (PN) replace the old picture numbers (PN).

When S pictures are not judged to be stored in the memory in step 1404,the picture numbers (PN) are regarded as “M”s in step 1407. The picturenumbers (PN) are not changed. In step 1405, it is judged whether or notthe pictures to be coded are stored in the memory. When the currentpictures are judged to be stored in the memory, the picture numbers (PN)are incremented to PN+1 in step 1408, and the incremented picturenumbers (PN) replace the old picture numbers (PN).

When the pictures to be coded are judged not to be stored in the memory,the picture numbers (PN) are not changed.

In step 1409, target pictures are coded. In step 1410, it is judgedwhether or not all the current pictures have been coded. When all thecurrent pictures have not been coded, step 1401 must be repeated. Whenall the current pictures have been coded, step 1401 finishes.

The processing shown in FIG. 13 makes it possible to produce coded datastreams whose picture numbers (PN) are continuous in the streams afterswitching coded pictures.

Also, the coded signals (Str) produced in this way can be decoded basedon the decoding method by using the picture decoding apparatus in thefifth embodiment. In this way, the picture decoding apparatus thatdecodes coded signals in the sixth embodiment is realized.

Also, the coding and the decoding methods shown in the above-mentionedfirst to sixth embodiments can be implemented in mobile communicationinstruments such as cellular phones and car navigation systems and incameras such as digital video cameras or digital steel cameras by usingsemiconductors such as LSI. Also, there are 3 types of apparatuses whichare suitable for the implementation: a sending and receiving typeterminal with both of a coding and a decoding apparatuses, a sendingterminal with only a coding apparatus, and a receiving terminal withonly a decoding apparatus.

Seventh Embodiment

Pictures that should be referred to by the pictures to be decoded areshown in a form of picture numbers (PN). Also, errors of picture numbers(PN) can be detected based on the increase and the decrease in picturenumbers (PN). FIG. 14 shows the processing of checking and correctingerrors of picture numbers (PN) based on picture numbers (PN).

First, picture numbers (PN) are detected in step 20. Next, picture types(PicType) are detected in step 21. Then, it is judged in step A1 whetheror not picture numbers (PN) detected in step 20 are continuous. When thepicture numbers (PN) are judged to be continuous in step A2, errorchecking and correcting processing of picture numbers (PN) is completed.When the picture numbers (PN) are not judged to be continuous in stepA2, errors must be corrected in step A3. Note that the processing ofchecking the stored maximum PN and reassigning “PN”s can be performedeither by being solely done after this error checking and correctingprocessing or by being concurrently done with this error checking andcorrecting processing.

The first conceivable method of error correcting processing in step A3is requesting for resending the data concerning the picture numbers witherrors and following the error checking processing of picture numbers(PN) again after receiving the resent data. However, the cause of thediscontinuity in picture numbers (PN) of S pictures is not atransmission error. That is, as the number of pictures in each streamthat is stored in the memory prior to the S pictures may vary in thecase where the discontinuity in picture numbers (PN) of S pictures isfound, the picture whose picture number (PN) is required to be resentmay not exist, and thus it is highly unlikely that a missing picture canbe sent. Therefore, the requests for the pictures that cannot be sentare made endlessly, which may be a hindrance in displaying pictures. Forthis reason, as to countermeasures in the case of problems withdisplaying pictures, further explanation will be made in a tenthembodiment below.

Also, if the number of pictures in the stream after switching in thememory at the time of switching streams is not the same as that of thepictures in the original stream, displaying the pictures may not be doneproperly.

First, there are memories of a FIFO memory for short-time storage and amemory for long-time storage that can directly specify storage placeswithout employing “first-in, first-out” so as to store pictures longerthan the above-mentioned memory for short-time storage. When the memoryfor short-time storage can store 7 pictures and the memory for long-timestorage can store 4 pictures, one or more reference pictures arespecified based on the order calculated from the memory for short-timestorage. For example, used LT2 can be specified as the eighth picture(ldx=7) in the memory for long-time storage. In this way, referencepictures are specified based on the relative positions.

When there are 3 streams as shown in FIG. 7, the locations in the memoryto specify the identical pictures (such as S pictures shown in FIG. 7)vary from stream to stream as shown in FIG. 2. Further, when referringto pictures in another stream from S pictures, the locations in thememory to specify reference pictures vary according to the memory ineach stream. When a plurality of streams exist at the time of switchingstreams from a predetermined one to another stream, S pictures are thosepictures whose precedent pictures in the streams before and afterswitching are identical.

Not only S pictures but also I pictures can be the pictures at whichstreams are switched when the pictures that are decoded in a pluralityof streams in the reference memory are exactly the same, and thus Ipictures can be used for the same purpose as S pictures (for switchingstreams).

Considering various conditions like this, it is difficult to specifyreference pictures correctly if the number of pictures varies fromstream to stream, and it is highly likely that there emerges any erroreven when the reference pictures are specified.

Therefore, this seventh embodiment will show the coding and decodingmethods of additional information which is used for avoiding an endlesserror checking processing of picture numbers (PN) which is triggered byproblems like a discontinuity of picture numbers (PN) or an incongruityin the content of the memory. This additional information (all picturedelete information) is the order for showing that pictures, except for Ipictures and S pictures to be coded, must be deleted from the memory forreference in coding and decoding in order to prevent any error fromoccurring in the process of coding pictures after coding I picturescapable of intra-picture coding and the above-mentioned S pictures.

This process makes each memory condition identical in a plurality ofstreams after switching streams from a predetermined stream to anotherstream and makes it possible to correctly specify predetermined picturesin the memory even when reference pictures are needed forintra-predictive coding and so on. Also, preventing discontinuity inpicture numbers (PN) from being detected and corrected as an errorresolves the problem of hindering decoding which is caused by requestsfor resending nonexistent pictures.

The coding method will be explained below in FIG. 16A. FIG. 16A showsthe making processing of coded signals in this seventh embodiment.

First, picture numbers (PN) are detected in step 20. Next, picture types(PicType) are detected in step 21. In step A1, it is judged whether ornot the detected picture types are I pictures. When the detected picturetypes are I pictures, all the pictures except for the I pictures to becoded in the memory are deleted in step A10. In the following step A11,all picture delete information, which means deleting all the pictures inthe memory, is coded, and this is the last procedure of codingadditional information.

Also, step A1 in FIG. 16A can be used as a step for judging whether ornot the picture types are S pictures in the same coding processing asshown in FIG. 16B. Also, it is possible to combine step A1 with step A2and judge whether the picture types are I pictures or S pictures afterchecking the picture types in step 21.

As shown in FIG. 17A, when the picture types to be coded are proved tobe I pictures in step A1 at the time of checking the picture types instep 21, and when the picture numbers (PN) are judged to bediscontinuous after the judgment on whether or not the picture numbersare continuous similar to the judgment made in step A3, it is alsopossible to delete all the pictures except for the I pictures to becoded in the memory. On the other hand, when picture numbers (PN) arejudged to be continuous in step A3, the pictures in the memory are notdeleted. When checking S pictures as picture types, the same explanationshown in FIG. 17A holds true. Also, it is possible to combine step A1with step A2 and judge whether the picture types are I pictures or Spictures after checking the picture types in step 21.

It is also possible to perform the processing of A30, as shown in FIG.17B, to judge whether or not the numbers of pictures stored in thememory are the same so as to avoid the occurrence of errors caused bythe difference in the number of pictures stored in the memory after theprocessing of step A3 shown in FIG. 17A. It is also possible to performthe processing of step A30 before going on to the processing of step A3shown in FIG. 17. Further, it is possible to delete all the picturesafter performing Step A30 before performing Step A3 in FIG. 17B when thenumbers of pictures vary from stream to stream, and it is possible todelete all the pictures shown in step A10 when the numbers of picturesdo not vary from stream to stream and the picture numbers (PN) arediscontinuous. (FIG. 18)

In this way, the processing of FIG. 17 makes it possible to keep storingas many pictures as possible that may be reference pictures in thememory and to improve the reproducibility of pictures decreasing errors.As to I pictures or S pictures, they need no error correction when thenumbers of pictures stored in the memory vary or the picture numbers(PN) are discontinuous, which simplifies memory control in the codingapparatus.

Note that specifying I pictures and ordering deletion of all thepictures in the memory can be shown by picture types that specifiesspecial pictures as I pictures.

Eight Embodiment

FIG. 19 is a block diagram showing the structure of the picture codingapparatus of the present invention. The block diagram of the picturecoding apparatus in the present invention shown in FIG. 19 is an examplefor realizing the picture coding method in FIG. 16.

A picture number generating unit (PNGen) generates picture numbers (PN).Picture numbers (PN) are IDs that identify pictures that are stored inthe reference picture memory (Mem), and each different picture that isstored in the reference picture memory (Mem) is given an exclusivepicture number (PN). Usually, picture numbers (PN) are incremented by 1each time a picture is stored in the reference picture memory. Ifpicture numbers (PN) received in the picture decoding apparatus areincremented by 2 or more, it is possible to detect the lack of picturesto be stored in the transmission line by the picture decoding apparatusand to carry out error correction processing such as picture improvement(making the error less conspicuous) or error correction (retransmittingthe picture with no errors to reproduce the picture).

When the picture type information (PicType) shows that the picturesconcerned are S pictures (corresponding to the processing of step A2 inFIG. 16), a picture deleting unit (PicDel 3) orders the referencepicture memory (Mem) to delete the pictures that are stored in thereference picture memory (Mem) except for the pictures to be coded, andsends the information to a variable length coding unit (VLC) at the sametime.

On the other hand, when the picture type information (PicType) showsthat the pictures concerned are I pictures (corresponding to theprocessing of step A1 in FIG. 16), the picture deleting unit (PicDel 3)orders the reference picture memory (Mem) to delete the pictures thatare stored in the reference picture memory (Mem) except for the picturesto be coded, and also sends the information to the variable lengthcoding unit (VLC) at the same time.

A picture encoding unit (PicEnc) refers to the pictures that are storedin the reference picture memory (Mem) so as to code the input picturesignal (Vin) including frequency conversion and quantization as picturetypes which are shown by picture type information (PicType), and sendsthe result to the picture decoding unit (PicDec) or the variable lengthcoding unit (VLC).

A picture decoding unit (PicDec) inversely quantizes and converts thefrequency of the information which is coded in the picture encoding unit(PicEnc) as picture types which are shown as picture type information(PicType), and stores the information in the reference picture memory(Mem) as picture numbers (PN) so as to refer the picture numbers in thefollowing picture coding.

The variable length coding unit (VLC) performs variable length coding onthe information which is coded in the picture encoding unit (PicEnc) soas to make a bit stream, and codes necessary information for decodingsuch as the information for deleting pictures that are stored in thereference picture memory (Mem) notified by picture deleting unit (PicDel3), picture numbers (PN), and picture type information (PicType) tooutput the information as coded signals (Str).

The structure of the coded signals (Str) in the present invention willbe shown in FIGS. 10C and 10D. The data will be explained below.

First, Picture numbers (PN) are coded. Then, in the following order,information for deleting pictures that are stored in the referencepicture memory (Mem), picture type information (PicType) and picturecoding data outputted by the in picture encoding unit (PicEnc) arelocated.

As FIG. 10C is merely an example of data location, it is possible toswitch data coding orders as shown in FIG. 10D.

The above-mentioned processing enables the picture coding apparatus torealize the picture coding method shown in FIG. 16 and to provide acoding apparatus with high error-resistance.

Ninth Embodiment

FIG. 20 is a block diagram showing the structure of the picture codingapparatus of the present invention. The block diagram of the picturecoding apparatus of the present invention shown in FIG. 20 is an examplefor realizing the picture coding method in FIG. 17. An explanationconcerning the same units as those explained with reference to FIG. 19will be omitted from the following explanations.

FIG. 20 and FIG. 19 differ in the process of the picture deleting unit(PicDel 4). To be specific, when the picture type information (PicType)shows that the pictures concerned are S pictures (corresponding to theprocessing of step A2 in FIG. 17) and the number of pictures varies fromstream to stream when comparing the picture numbers, (corresponding tothe processing of step A30 in FIG. 17), the picture deleting unit(PicDel 4) orders the reference picture memory (Mem) to delete thepictures that are stored in the reference picture memory (Mem) exceptfor the pictures to be coded, and sends the information to the variablelength coding unit (VLC) at the same time. The same is done in the caseof I pictures which are shown by the picture type information (PicType).The structure of coded signals of the present invention is the same asthe ones shown in FIGS. 10C and 10D.

The above-mentioned processing enables the picture coding apparatus torealize the picture coding method shown in FIG. 17 and to provide thecoding apparatus with a high error-resistance.

Tenth Embodiment

The seventh embodiment described above showed that picture display maybe hindered because requests for resending pictures that cannot beresent are repeatedly made when a discontinuity of picture numbersoccurs at an S picture. The problem-solving method for the example ofpicture display with problems that are caused by this reason will beexplained below.

FIG. 21A shows the processing of decoding the coded pictures.

First, picture numbers (PN) are detected in step 20. Next, picture types(PicType) are detected in step 21. In step A1, it is judged whether ornot the detected picture types are I pictures. When the detected picturetypes detected are not I pictures, it is judged in step A3 whether thepicture numbers (PN) are continuous. On the other hand, when the picturetypes detected are I pictures, there is no need to detect or correcterrors and a series of processing finishes.

When the picture numbers (PN) are not judged to be continuous in stepA3, errors are corrected in step A4. On the other hand, when the picturenumbers (PN) are judged to be continuous in step A3, the error check andcorrection processings are completed.

Error correction in step A4 can be, for example, a processing ofdetecting the stored maximum picture numbers (PN) as explained in theabove-mentioned embodiments, or deleting all the pictures that are inthe memory after receiving the all picture delete information, whichmeans deleting all the pictures in the memory, in the processing ofreassigning picture numbers (PN).

As shown in FIG. 21B, of step A1 in FIG. 21A, the same coding processingcan be performed as a step of judging whether the picture types are Spictures or not. Also, it is possible to combine step A1 with step A2and judge which picture types of I pictures or S pictures they haveafter checking the picture types in step 21.

In this way, it is possible to avoid hindering decoding as a result ofrepeating requests for resending the missing pictures so as to correcterrors when picture numbers of I pictures or S pictures are notcontinuous. This processing at an I picture is especially useful in thecase of a special I picture capable of switching streams.

Eleventh Embodiment

FIG. 22 is a block diagram showing the structure of the picture decodingapparatus of the present invention. The block diagram of the picturedecoding apparatus of the present invention shown in FIG. 22 shows anexample for realizing the picture decoding method shown in FIG. 21. Anexplanation concerning the same units as explained with reference toFIG. 11 will be omitted from the following explanations.

The different point between FIG. 22 and FIG. 11 is the processing in theerror checking unit (ErrChk) using picture types (PicType) by thepicture number checking unit (PNchk). To be specific, when picturenumbers (PN) to be inputted in a picture number checking unit (PNchk)are not continuous and picture types are neither I pictures nor Spictures, an error correction order (Err) is outputted by the errorchecking unit (ErrChk). With an error correction order, processing suchas the stored maximum PN check, PN reassignment or deleting all thepictures in the memory according to the all picture delete information,which means deleting all the pictures in the memory.

The above-mentioned processing enables the picture coding apparatus torealize the picture coding method shown in FIG. 21 and to provide adecoding apparatus with a high error-resistance.

Twelfth Embodiment

The twelfth embodiment explains another countermeasure against anendless error check of picture numbers (PN) which is caused by problemssuch as a discontinuity of picture numbers (PN) or an incongruity inmemory contents. This twelfth embodiment differs from the seventhembodiment in that the picture numbers (PN) are reassigned from “0” whenall the pictures are deleted after the same step of deleting all thepictures in the coding shown in the seventh embodiment.

This process makes the respective memory conditions in a plurality ofstreams identical after switching streams from a predetermined stream toanother stream, and initializes the picture numbers (PN). Thus, thisprocess makes it possible to correctly specify the predeterminedpictures in the memory even when reference pictures are required inintra-predictive coding and so on. It is also possible to resolve theproblem of hindering decoding when switching the coded streams to bedecoded from the predetermined stream to another stream by avoidingcorrecting any picture number discontinuity as errors.

As explained, each picture in the stream is given an exclusive picturenumber (PN) that is continuous in display time order in the codedstreams which are obtained by coding moving pictures. The reason why thepicture numbers (PN) are given exclusive picture numbers (PN) that arecontinuous in display time order is that it makes it possible to checkthe lack of pictures in the coded streams which is caused by an error inthe transmission line in the case where the picture decoding apparatusreceives the coded streams by way of the transmission line. When apicture number (PN) of a picture to be inputted in the display timeorder is incremented by 2 or more while the received coded stream isbeing decoded, this picture decoding apparatus can check whether atransmission error occurred just before the receiving time of thepicture concerned and can request the sender to resend the missingpictures. Therefore, as long as the picture decoding apparatus isdecoding one coded stream continuously, the picture decoding apparatuscan check transmission errors effectively and receive the resent missingpictures so as to decode the coded streams perfectly.

However, this error check causes a glitch of endless error checkingprocessing, in the case where the picture decoding apparatus is used forcontinuing decoding after switching to another coded stream with adifferent picture rate while decoding a coded stream after inputting aplurality of coded streams which are obtained by coding the same movingpicture at a different picture rate. The cause of this glitch is thatpicture numbers (PN) of pictures except for the first picture in eachstream vary among coded pictures with different picture rates even inthe case of the pictures to be displayed at the same time, in otherwords, picture numbers (PN) are continuous within each coded stream inthe display time order. Therefore, when the decoding target is switchedto another stream in the middle of decoding one coded stream in apicture decoding apparatus, picture numbers (PN) are discontinuous evenin the case of the pictures to be displayed at the same time. The codingmethod using additional information (all picture delete information) hasbeen explained in the seventh embodiment so as to avoid the endlesserror check of picture numbers (PN) caused by the problems such as adiscontinuity of picture numbers (PN) or an incongruity of memorycontents in this way. This additional information is the order fordeleting all the pictures, except for pictures to be coded, from thememory for reference in coding or decoding so that any error should notoccur at the time of switching streams in the process of picture codingafter coding I pictures for intra-coding or the above-mentioned Spictures.

A coding method of the present invention will be explained below withreference to FIG. 23. FIG. 23A shows the making processing of codedsignals in this twelfth embodiment.

First, picture numbers (PN) are detected in step 1. Next, picturenumbers (PN) which are detected in step 1 are coded in step 2. Then,picture types (PicType) are detected in step 3. In step 3, it is judgedwhether or not the detected picture types are S pictures.

When the detected picture types are S pictures, all picture deleteinformation, which means deleting all the pictures in the memory, iscoded in step 5. Next, S pictures are coded in step 6A. Then, picturenumbers are initialized in step 7, and in the following step 8, all thepictures, except for the S pictures to be coded, in the memory aredeleted in step 8. Up to this point, the processing of coding additionalinformation and initializing picture numbers (PN) finishes.

As picture numbers (PN) are continuous unless the detected picture typesare S pictures, these pictures are coded in step 6B, but the processingis completed without coding additional information, initializing picturenumbers (PN), and deleting all the pictures.

The initialization of picture numbers (PN) in step 7 is performed, forexample, by giving a picture number of “0” to the coded S pictures. Inother words, initializing picture numbers (PN) of S pictures meansgiving the pictures after S pictures in the display time order newpicture numbers (for example, PN 1) starting from the picture number (PN0) of S pictures. Consequently, picture numbers (PN) are initializedafter coding S pictures (that is, after coding the picture numbers (PN)of S pictures).

The judgment on whether or not the pictures are S pictures is made instep 4, and the judgment on whether or not the pictures are I picturescan also be made. Also, when there is a step of deleting all thepictures in step 23A, picture numbers (PN) should be initializedconcurrently because judging whether or not the pictures are I picturesor S pictures is not the sole determinative standard for whether picturenumbers (PN) should be initialized. Also, picture number (PN) codingprocessing in Step 2 can be performed at any time between picture numberchecking processing in step 1 and the picture number initializingprocessing in step 7. Also, it is possible to initialize picture numbers(PN) in step 7 after deleting all the pictures, except for the Spictures to be coded, in the memory in step 8. Also, the processing ofcoding all picture delete information, which means deleting all thepictures in the memory, in step 5 is the processing after the judgmentas to whether or not the pictures are S pictures and can be performed atany time before the processing shown in FIG. 23A is finished. Also, itis possible not to code additional information by using special picturetypes (PicType) that include additional information which indicates thatall the pictures except for the pictures to be coded are to be deletedfrom the memory for reference in coding or decoding. Reassigning picturenumbers (PN) so as to switch streams at S pictures or I pictures iseffective, but the effectiveness is not limited to the case wherestreams are switched at S pictures or I pictures, in other words,picture numbers (PN) can be reassigned in the same way on the conditionthat other pictures such as P pictures are capable of switching streamsand that there is a step of deleting all the pictures which areunnecessary for reference.

FIG. 24 is a block diagram showing the structure of the picture codingapparatus which is capable of realizing the coding method in the twelfthembodiment.

A picture number generating unit (PNGen 2) generates Picture numbers(PN). Picture numbers (PN) are IDs that identify the pictures that arestored in the reference picture memory (Mem), where each picture that isstored in the reference picture memory (Mem) is given an exclusivepicture number (PN). Basically, picture numbers (PN) are incremented by1 each time a picture is stored in the reference picture memory (Mem).Also, picture numbers (PN) of the current S pictures are initialized to“0” after S pictures are coded according to the notification from apicture encoding unit (PicEnc).

When picture type information (PicType) shows that the pictures are Spictures (corresponding to the processing of step 3 in FIG. 23), apicture deleting unit (PicDel 5) notifies the reference picture memory(Mem) of an order of deleting the pictures except for the pictures to becoded (all picture delete information) that are stored in the referencepicture (Mem) memory, and notifies the variable length coding unit (VLC)of the information at the same time.

The picture encoding unit (PicEnc) refers to the pictures that arestored in the reference picture memory (Mem) so as to code the inputpicture signal (Vin) into picture types shown by the picture typeinformation (PicType) while concurrently performing frequency conversionand quantization, and sends the result to the picture decoding unit(PicDec) and the variable length coding unit (VLC). Also, the pictureencoding unit (PicEnc) notifies the picture number generating unit(PNGen 2) of the order of initializing picture numbers (PN) after codingS pictures.

The picture decoding unit (PicDec) inversely quantizes and inverselyconverts the information which is coded in the picture encoding unit(PicEnc) into picture types that are shown as picture type information(PicType), and stores the information in the reference picture memory(Mem) by associating the information with picture numbers (PN) so as torefer to the picture types in the following picture coding.

The variable length coding unit (VLC) performs variable length coding onthe information which is coded in picture encoding unit (PicEnc) so asto make a bit stream, and codes necessary information in decoding suchas the information for deleting pictures that are stored in thereference picture memory (Mem) (that is, all picture delete information)which is notified by picture deleting unit (PicDel 5), picture numbers(PN), and picture type information (PicType) to output the informationas coded signals (Str).

Next, a decoding method will be explained below with reference to FIG.23B. FIG. 23B shows the decoding processing of coded signals.

First, picture numbers (PN) are decoded in step 9. Next, it us judgedwhether or not all picture delete information is coded in step 10.

When the all picture delete information is judged to be coded in step10, the all picture delete information is decoded in step 11. Then,pictures are decoded in step 12A. Thereafter, all the pictures exceptfor the pictures to be decoded in the memory are deleted in step 13, andpicture numbers (PN) are initialized in step 14. Up to this point, theprocessing of decoding additional information and the initialization ofpicture numbers (PN) finishes.

When the all picture delete information is judged not to be coded instep 10, pictures are decoded in step 12B, and the processing ofdecoding additional information and the initializing of picture numbers(PN) is completed in step 12B.

The initialization of picture numbers (PN) in step 14 is performed, forexample, by giving a picture number of “0” to the decoded pictures. Inother words, when decoding the coded signals which are coded accordingto the coding procedure shown in FIG. 23A, initializing picture numbers(PN) of S pictures means giving the pictures after S pictures in thedisplay time order new continuous picture numbers starting from thepicture number of the S pictures.

When there is a step of deleting all the pictures in FIG. 23B, theprocessing of initializing picture numbers (PN) is necessary, in otherwords, the judgment as to whether the picture numbers should beinitialized is not influenced by the picture types that are to bedecoded. Also, the processing of initializing picture numbers (PN) instep 14 can be performed before the processing of deleting all thepictures except for the pictures to be coded in the memory in step 13.It is possible not to code additional information by using specialpicture types (PicType) that include additional information indicatingthat all the pictures except for the pictures to be decoded are to bedeleted from the memory for reference in decoding.

FIG. 25 is a block diagram showing the structure of the picture decodingapparatus that realizes the decoding method of this twelfth embodiment.

A variable length decoding unit (VLD) decodes coded signals (Str), andoutputs various information (such as an order for deleting the picturesthat are stored in the reference picture memory (Mem), picture typeinformation (PicType), picture numbers (PN), information for reassigningpicture numbers (PN), and picture data.)

First, the order for deleting the pictures that are stored in thereference picture memory (Mem) which is obtained in the variable lengthdecoding unit (VLD) (all picture delete information) is sent to apicture deleting unit (PicDel 6). Next, the picture deleting unit(PicDel 6) deletes the specified pictures that are stored in thereference picture memory (Mem).

Picture type information (PicType) which is obtained in the variablelength decoding unit (VLD) is sent to a picture decoding unit (PicDec)so as to specify the decoding method.

Picture numbers (PN) which are obtained in the variable length decodingunit (VLD) are sent to the reference picture memory (Mem) to be used aspicture numbers (PN) at the time of storing the decoded pictures in thepicture decoding unit (PicDec).

All picture delete information obtained in the variable length decodingunit (VLD) is sent to the picture number changing unit (PNchg 2). Thepicture number changing unit (PNchg 2) reassigns (initializes) picturenumbers (PN) of the pictures that are stored in the reference picturememory (Mem) according to the order. To be specific, after all thepictures except for the pictures to be decoded (S pictures) in thereference picture memory (Mem), the picture number changing unit (PNchg2) reads out the picture numbers (PN) of the pictures that are stored inthe reference picture memory (Mem), changes the values of the read-outpicture numbers (PN) to “0”, and writes the picture numbers (PN) in thereference picture memory (Mem).

Picture data obtained in the variable length decoding unit (VLD) aredecoded using the decoding method based on the picture type which isshown by the picture type information (PicType) in the picture decodingunit (PicDec). In other words, P pictures and B pictures are decoded byreferring to the pictures that are stored in the reference picturememory (Mem), whereas I pictures are decoded without referring topictures that are stored in the reference picture memory. The decodedpictures obtained in this manner are stored in the reference picturememory (Mem) and are outputted as decoded picture signals (Vout).

The above-mentioned structure makes it possible to realize the picturedecoding apparatus, to realize the picture decoding method shown in FIG.23, and to provide a decoding apparatus with a high error-resistance.

The processing of the coding and the decoding methods shown in thistwelfth embodiment makes the memory conditions in a plurality of streamsidentical after switching streams from a predetermined stream to anotherstream, and thus makes it possible to correctly specify thepredetermined pictures in the memory even when reference pictures arerequired in intra-predictive coding and so on.

It is possible to change I pictures into special pictures at whichstreams can be reproduced by deleting all the pictures in the referencememory when using I pictures, while the above-mentioned embodimentexplains that additional information (all picture delete information)and picture types (PicType) can be coded all at once. These special Ipictures are called Instantaneous Decoder Refresh (IDR) pictures. An IDRpicture is effective as a leading I picture of a Group of Pictures (GOP)because IDR pictures become a starting position of random access. Bydetermining that all the pictures except for the current pictures in thememory are deleted and that picture numbers (PN) are initialized aftercoding the current pictures each time these IDR pictures are coded,there is no need to code additional information even when all thepictures except for the current pictures in the memory are deleted inthe picture decoding apparatus. In this case, the picture decodingapparatus detects IDR pictures in the coded streams based on picturetypes, deletes all the pictures except for the current IDR pictures inthe memory, and initializes picture numbers (PN) after coding anddecoding the current IDR pictures even when any additional informationis not coded each time IDR pictures are decoded.

Thirteenth Embodiment

Moreover, storing programs to realize the structures of the picturecoding and the decoding methods shown in the above-mentioned embodimentson a storage medium such as a flexible disc makes it possible to easilyperform the processing shown in the above-mentioned embodiments in anindependent computer system.

FIG. 26 is an illustration concerning the storage medium for storing theprogram to realize the coding and the decoding methods shown in thefirst to the twelfth embodiments mentioned above in the independentcomputer systems.

FIG. 26B shows a flexible disc and a front view and a cross-sectionalview of the appearance of the flexible disc, and FIG. 26A shows anexample of a physical format of a flexible disc as a recording mediumbody. A flexible disc (FD) is contained in a case F, a plurality oftracks (Tr) are formed concentrically on the surface of the disc fromthe periphery of the disc into the inner radius of the disc, and eachtrack is divided into 16 sectors (Se) in the angular direction.Therefore, in the case of the flexible disc (FD) storing theabove-mentioned program, the picture coding method and the picturedecoding method, as the program, is recorded in an area which isallocated for it on the flexible disc (FD).

Also, FIG. 26C shows the structure for recording and reading out theprogram on the flexible disc (FD). When the program is recorded on theflexible disc (FD), the computer system (Cs) writes in the picturecoding method or the picture decoding method as a program via a flexibledisc drive (FDD). When the picture coding method and the decoding methodmentioned above are constructed in the computer system by the program onthe flexible disc, the program is read out from the flexible disc drive(FDD) and transferred to the computer system (Cs).

The above explanation is made by using a flexible disc as a recordingmedium, but the same processing can also be performed by using anoptical disc. In addition, the recording medium is not limited toflexible discs and optical discs, in other words, any other medium whichis capable of recording a program such as CD-ROMs, memory cards, and ROMcassettes can be used.

Here, the applications of the picture coding method and the picturedecoding method shown in the above-mentioned embodiment and the systemusing them will be further explained.

FIG. 27 is a block diagram showing the overall configuration of acontent supply system ex100 for realizing a content distributionservice. The area for providing the communication service is dividedinto cells of desired sizes, and cell sites ex107-ex110 of fixedwireless stations are placed in the respective cells.

This content supply system ex100 is connected to apparatuses such as acomputer ex111, a Personal Digital Assistant (PDA) ex112, a cameraex113, a cell phone ex114 and a cell phone with a camera ex115 via, forexample, a combination of the Internet ex101, an Internet serviceprovider ex102, a telephone network ex104 and cell sites ex107-ex110.

However, the content supply system ex100 is not limited to theconfiguration as shown in FIG. 27, and may be connected to a combinationof any of the illustrated components. Also, each apparatus can beconnected directly to the telephone network ex104, instead of beingconnected through the cell sites, as fixed radio stations ex107-ex110.

The camera ex113 is a apparatus which is capable of shooting video(moving pictures) such as a digital video camera. The cell phone can bea cell phone of a Personal Digital Communication (PDC) system, a CodeDivision Multiple Access (CDMA) system, a Wideband-Code DivisionMultiple Access (W-CDMA) system or a Global System for MobileCommunications (GSM) system, and a Personal Handy-phone system (PHS) orthe like.

A streaming server ex103 is connected to the camera ex113 via thetelephone network ex104 and the cell site ex109, which enables livedistribution or the like using the camera ex113 based on the coded datathat is transmitted from the user. Either the camera ex113 or the serverfor transmitting the data can code the shot (captured) data. Also, themoving picture data which is shot by the camera ex116 can be transmittedto the streaming server ex103 via the computer ex111. The camera ex116is a apparatus which is capable of shooting still and moving picturessuch as a digital camera. Either the camera ex116 or the computer ex111can code the moving picture data. An LSI (large scale integratedcircuit) ex117 which is included in the computer ex111 or the cameraex116 performs coding processing. Software for coding and decodingpictures can be integrated into any type of storage media (such asCD-ROMs, flexible discs and hard discs) that is a recording medium whichis readable by the computer ex111 or the like. Furthermore, a cell phonewith a camera ex115 can transmit the moving picture data. This movingpicture data is the data that is coded by the LSI included in the cellphone ex115.

The contents supply system ex100 codes contents (such as a music livevideo) which are shot by users using the camera ex113, the camera ex116or the like in the same manner as the above-mentioned embodiments andtransmits the coded contents to the streaming server ex103, while thestreaming server ex103 makes stream distribution of the contents data tothe clients upon their request. The clients include the computer ex111,the PDA ex112, the camera ex113, the cell phone ex114 and so on whichare capable of decoding the above-mentioned coded data. In the contentssupply system ex100, the clients can thus receive and reproduce thecoded data, and further can receive, decode and reproduce the data inreal time so as to realize personal broadcasting in this way.

When each apparatus in this system performs coding or decoding, thepicture coding apparatus or the picture decoding apparatus can be used,as shown in the above-mentioned embodiments.

A cell phone will be explained as an example of the apparatus.

FIG. 28 is a diagram showing the cell phone ex115 using the picturecoding method and the picture decoding method explained in theabove-mentioned embodiments. The cell phone ex115 has an antenna ex201for communicating with the cell site ex110 via radio waves, a cameraunit ex203 which is capable of shooting moving and still pictures suchas a CCD camera, a display unit ex202 such as a liquid crystal displayfor displaying the data that is obtained by decoding pictures and thelike which are shot by the camera unit ex203 and received by the antennaex201, a body unit including a set of operation keys ex204, a voiceoutput unit ex208 such as a speaker for outputting voices (audio), avoice input unit 205 such as a microphone for inputting voices (audio),a storage medium ex207 for storing coded or decoded data such as data ofmoving or still pictures which are shot by the camera, data of receivede-mail and data of moving or still pictures, and a slot unit ex206 forattaching the storage medium ex207 to the cell phone ex115. The storagemedium ex207 is equipped with a flash memory element, which is a kind ofElectrically Erasable and Programmable Read Only Memory (EEPROM) that isan electrically erasable and rewritable nonvolatile memory, in a plasticcase such as SD cards.

Next, the cell phone ex115 will be explained with reference to FIG. 29.In the cell phone ex115, a main control unit ex311 for overallcontrolling each unit of the body unit including the display unit ex202and operation keys ex204 is connected to a power supply circuit unitex310, an operation input control unit ex304, a picture coding unitex312, a camera interface (I/F) unit ex303, an Liquid Crystal Display(LCD) control unit ex302, a picture decoding unit ex309, ademultiplexing unit ex308, a recording and reproducing unit ex307, amodem circuit unit ex306 and a voice processing unit ex305 to each othervia a synchronous bus ex313.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex310 supplies respective components withpower from a battery pack so as to activate the digital cell phone witha camera ex115 for making it into a ready state.

In the cell phone ex115, the voice processing unit ex305 converts thevoice signals that are received by the voice input unit ex205 inconversation mode into digital voice data under the control of the maincontrol unit ex311 including a CPU, a ROM and a RAM, the modem circuitunit ex306 performs spread spectrum processing of the digital voicedata, and the communication circuit unit ex301 performsdigital-to-analog conversion and frequency transformation of the data soas to transmit it via the antenna ex201. Also, in the cell phone ex115,the communication circuit unit ex301 amplifies the data that is receivedby the antenna ex201 in the conversation mode and performs frequencytransformation and analog-to-digital conversion for the data, the modemcircuit unit ex306 performs inverse spread spectrum processing of thedata, and the voice processing unit ex305 converts it into analog voicedata so as to output it via the voice output unit 208.

Furthermore, when transmitting e-mail in the data communication mode,the text data of the e-mail which is inputted by operating the operationkeys ex204 on the body unit is sent out to the main control unit ex311via the operation input control unit ex304. In the main control unitex311, after the modem circuit unit ex306 performs spread spectrumprocessing of the text data and the communication circuit unit ex301performs digital-to-analog conversion and frequency transformation forthe data, the data is transmitted to the cell site ex110 via the antennaex201.

When picture data is transmitted in the data communication mode, themoving picture data which is shot by the camera unit ex203 is suppliedto the picture coding unit ex312 via the camera interface unit ex303.When the picture data is not transmitted, it is also possible to displaythe picture data which is shot by the camera unit ex203 directly on thedisplay unit 202 via the camera interface unit ex303 and the LCD controlunit ex302.

The picture coding unit ex312, which includes the picture codingapparatus as explained in the present invention, compresses and codesthe picture data that is supplied from the camera unit ex203 by usingthe coding method which is used for the picture coding apparatus asshown in the above-mentioned embodiments so as to transform the picturedata into coded picture data, and sends the coded picture data out tothe demultiplexing unit ex308. At this time, the cell phone ex115 sendsout the voices which are received by the voice input unit ex205 duringshooting by the camera unit ex203 to the demultiplexing unit ex308 asdigital voice data via the voice processing unit ex305.

The demultiplexing unit ex308 multiplexes the coded picture data that issupplied from the picture coding unit ex312 and the voice data that issupplied from the voice processing unit ex305 by using a predeterminedmethod, the modem circuit unit ex306 performs spread spectrum processingof the multiplexed data which is obtained as a result of themultiplexing, and the communication circuit unit ex301 performsdigital-to-analog conversion and frequency transformation of the datafor transmitting via the antenna ex201.

As for receiving data of a moving picture file which is linked to a Webpage or the like in the data communication mode, the modem circuit unitex306 performs spread spectrum processing of the data that is receivedfrom the cell site ex110 via the antenna ex201, and sends out themultiplexed data which is obtained as a result of the processing to thedemultiplexing unit ex308.

In order to decode the multiplexed data that is received via the antennaex201, the demultiplexing unit ex308 separates the multiplexed data intoa bit stream of picture data and a bit stream of voice data, andsupplies the current coded picture data to the picture decoding unitex309 and the current voice data to the voice processing unit ex305,respectively, via the synchronous bus ex313.

Next, the picture decoding unit ex309, which includes the picturedecoding apparatus as explained in the present invention, decodes thebit stream of picture data by using the decoding method corresponding tothe coding method as shown in the above-mentioned embodiments so as togenerate reproduced moving picture data, and supplies this data to thedisplay unit ex202 via the LCD control unit ex302, and thus, the movingpicture data which is included in a moving picture file that is linkedto a Web page, for instance, is displayed. At the same time, the voiceprocessing unit ex305 converts the voice data into analog voice data,and supplies this data to the voice output unit ex208, and thus, voicedata which is included in a moving picture file that is linked to a Webpage, for instance, is reproduced.

The present invention is not limited to the above-mentioned system, andat least one of the picture coding apparatus or the picture decodingapparatus in the above-mentioned embodiments can be incorporated into adigital broadcasting system as shown in FIG. 30. Such ground-based orsatellite digital broadcasting has been in the news lately. Morespecifically, a bit stream of video information is transmitted from abroadcast station ex409 to a communication or a broadcast satelliteex410 via radio waves. Upon receipt of the bit stream, the broadcastsatellite ex410 transmits radio waves for broadcasting, a home-useantenna ex406 with a satellite broadcast reception function receives theradio waves, and a television (receiver) ex401 or a set top box (STB)ex407 decodes and reproduces the bit stream. The picture decodingapparatus as shown in the above-mentioned embodiments can be implementedin the reproduction apparatus ex403 for reading off and decoding the bitstream recorded on a storage medium ex402 that is a recording mediumsuch as a CD and a DVD. In this case, the reproduced moving picturesignals are displayed on a monitor ex404. It is also conceived toimplement the picture decoding apparatus in the set top box ex407connected to a cable ex405 for a cable television or the antenna ex406for satellite and/or ground-based broadcasting so as to reproduce themon a monitor ex408 of the television. The picture decoding apparatus maybe incorporated into the television, instead of in the set top boxex407. Alternatively, a car ex412 having an antenna ex411 can receivesignals from the satellite ex410 or the cell site ex107 for reproducingmoving pictures on a display apparatus such as a car navigation systemex413.

Furthermore, the picture coding apparatus shown in the above-mentionedembodiments can code picture signals for recording on a recordingmedium. As a concrete example, there is a recorder ex420 such as a DVDrecorder for recording picture signals on a DVD disc ex421 and a discrecorder for recording them on a hard disc. They can be recorded on anSD card ex422. If the recorder ex420 includes the picture decodingapparatus shown in the above-mentioned embodiment, the picture signalsthat are recorded on the DVD disc ex421 or the SD card ex422 can bereproduced for display on the monitor ex408.

Note that a conceivable structure of the car navigation system ex413 isthe structure without the camera unit ex203, the camera interface unitex303 and the picture coding unit ex312 that are existing components inFIG. 29. The same goes for the computer ex111, the television (receiver)ex401 and the like.

In addition, the following three types of implementations can beconceived for a terminal such as the above-mentioned cell phone ex114: asending/receiving terminal which is implemented with both an encoder anda decoder, a sending terminal which is implemented with only an encoder,and a receiving terminal which is implemented with only a decoder.

As described above, it is possible to use the picture coding method orthe picture decoding method in the above-mentioned embodiments in any ofthe above-mentioned apparatus and systems, and by using these methods,the effects explained in the above embodiments can be obtained.

From the present invention as described above, it will be obvious thatthe embodiments of the present invention may be varied in many ways.Such variations are not to be regarded as a departure from the spiritand scope of the present invention, and all such modifications whichwould be obvious to one skilled in the art are intended to be includedwithin the scope of the following claims.

INDUSTRIAL APPLICABILITY

The picture coding apparatus of the present invention is useful as apicture coding apparatus which is installed in personal computers withcommunication functions, PDAs, digital broadcasting stations andcellular phones.

In addition, the picture decoding apparatus of the present invention isuseful as a picture decoding apparatus which is installed in personalcomputers with communication functions, PDAs, STBs receiving digitalbroadcasting and cellular phones.

The invention claimed is:
 1. A picture decoding method for decodingcoded data stream comprising (i) a bit stream of picture typeinformation of a picture, (ii) a bit stream of a picture number of apicture, and (iii) a bit stream of all picture release information, theall picture release information being instructive to release allpictures stored in a memory, and (iv) a bit stream of coded image dataobtained by coding image data of a picture on a block by block basis,the picture decoding method comprising: obtaining picture typeinformation of a decoding target picture from the coded data stream;obtaining a picture number of the decoding target picture from the codeddata stream; determining whether all picture release information isincluded in the coded data stream; decoding a bit stream of coded imagedata of the decoding target picture on a block by block basis to obtaindecoded image data of the decoding target picture, the decoding targetpicture having a picture number; and storing the decoding target picturein a decoder memory, wherein, (1) when determined that the all picturerelease information is not included in the coded data stream, keepingthe picture number of the decoding target picture unchanged after thedecoding of the decoding target picture, decoding subsequent picturesfollowing the decoding target picture, the subsequent pictures havingconsecutive picture numbers following the picture number of the decodingtarget picture, and storing the subsequent pictures as a referencepicture in the decoder memory in a case that a picture number of thesubsequent picture increases by one in comparison with a picture numberof a preceding picture in decoding order, and (2) when determined thatthe all picture release information is included in the coded data streamand the all picture release information is not based on a picture type,releasing all pictures stored in a decoder memory except for thedecoding target picture, after the decoding of the decoding targetpicture, and prohibiting from using, as a reference picture, all thepictures except for the decoding target picture in decoding ofsubsequent pictures, changing the picture number of the decoding targetpicture to a different picture number of 0, after the decoding of thedecoding target picture and after the releasing of all the picturesexcept for the decoding target picture, decoding subsequent picturesfollowing the decoding target picture, the subsequent pictures havingconsecutive picture numbers following the different picture number ofthe decoding target picture, and storing the subsequent pictures as areference picture in the decoder memory in a case that a picture numberof the subsequent picture increases by one in comparison with a picturenumber of a preceding picture in decoding order, and (3) when determinedthat the all picture release information is included in the coded datastream and the all picture release information is based on a picturetype, releasing all pictures stored in the decoder memory except for thedecoding target picture, after the decoding of the decoding targetpicture, and prohibiting from using, as a reference picture, all thepictures except for the decoding target picture in decoding ofsubsequent pictures, decoding subsequent pictures following the decodingtarget picture, the subsequent pictures having consecutive picturenumbers following the picture number of the decoding target picture, andstoring the subsequent pictures as a reference picture in the decodermemory in a case that a picture number of the subsequent pictureincreases by one in comparison with a picture number of a precedingpicture in decoding order.
 2. The picture decoding method according toclaim 1, wherein all the pictures that are released are referencepictures.
 3. A picture decoding apparatus which decodes coded datastream comprising (i) a bit stream of picture type information of apicture, (ii) a bit stream of a picture number of a picture, and (iii) abit stream of all picture release information, the all picture releaseinformation being instructive to release all pictures stored in amemory, and (iv) a bit stream of coded image data obtained by codingimage data of a picture on a block by block basis, the picture decodingapparatus comprising: an obtainment unit operable to obtain picture typeinformation of a decoding target picture from the coded data stream andoperable to obtain a picture number of the decoding target picture fromthe coded data stream; a determining unit operable to determine whetherall picture release information is included in the coded data stream; adecoding unit operable to decode a bit stream of coded image data of thedecoding target picture on a block by block basis to obtain decodedimage data of the decoding target picture, the decoding target picturehaving a picture number; and a storing unit operable to store thedecoding target picture in a decoder memory, wherein, (1) whendetermined that the all picture release information is not included inthe coded data stream, the following units are performed: a keeping unitoperable to keep the picture number of the decoding target pictureunchanged after the decoding of the decoding target picture, a decodingunit operable to decode subsequent pictures following the decodingtarget picture, the subsequent pictures having consecutive picturenumbers following the picture number of the decoding target picture, anda reference picture storing unit operable to store the subsequentpictures as a reference picture in the decoder memory in a case that apicture number of the subsequent picture increases by one in comparisonwith a picture number of a preceding picture in decoding order, and (2)when determined that the all picture release information is included inthe coded data stream and the all picture release information is notbased on a picture type, the following units are performed: a releasingunit operable to release all pictures stored in a decoder memory exceptfor the decoding target picture, after the decoding of the decodingtarget picture, a prohibiting unit operable to prohibit from using, as areference picture, all the pictures except for the decoding targetpicture in decoding of subsequent pictures, a changing unit operable tochange the picture number of the decoding target picture to a differentpicture number of 0, after the decoding of the decoding target pictureand after the releasing of all the pictures except for the decodingtarget picture, a decoding unit operable to decode subsequent picturesfollowing the decoding target picture, the subsequent pictures havingconsecutive picture numbers following the different picture number ofthe decoding target picture, and a reference picture storing unitoperable to store the subsequent pictures as a reference picture in thedecoder memory in a case that a picture number of the subsequent pictureincreases by one in comparison with a picture number of a precedingpicture in decoding order, and (3) when determined that the all picturerelease information is included in the coded data stream and the allpicture release information is based on a picture type, the followingunits are performed: a releasing and prohibiting unit operable torelease all pictures stored in the decoder memory except for thedecoding target picture, after the decoding of the decoding targetpicture, and operable to prohibit from using, as a reference picture,all the pictures except for the decoding target picture in decoding ofsubsequent pictures, a decoding unit operable to decode subsequentpictures following the decoding target picture, the subsequent pictureshaving consecutive picture numbers following the picture number of thedecoding target picture, and a reference picture storing unit operableto store the subsequent pictures as a reference picture in the decodermemory in a case that a picture number of the subsequent pictureincreases by one in comparison with a picture number of a precedingpicture in decoding order.
 4. The picture decoding apparatus accordingto claim 3, wherein all the pictures that are released are referencepictures.